Resilience for Autonomous Agents

In this paper we show how the resilience approach can give a generic solution to the problems of looping and high-bandwidth output in autonomous agents. A resilient approach to looping is for the agent to delay responding again to a source that has recently triggered a task. A resilient approach to high-bandwith output is for the agent to delay output when the overall “noise” level in the environment is high. The conditions under which the delays are triggered may be determined by data on past system behaviour. Our generic approach allows agents to limit themselves, without requiring them to perform semantic analyses. 1 Why We Need Resilient Agents There is a general trend for computer systems to become larger, more complex, more heterogeneous, and more dynamically varying. As a result, central human control for many systems is becoming difficult or impossible. There is a need to perform some operations performed automatically and autonomously, in a distributed fashion. Agents, or pieces of software capable of some autonomous behaviour, are already used for tasks such as searching for online information, monitoring systems, and e-shopping. (To see examples of these, visit agentland.com or botspot.com, and see [1] for an overview.) In HP’s Utility Data Center, OpenView agents perform low-level tasks such as monitoring memory usage or trapping events [2]. For very large and complex systems, security and reliability are particularly important and problematic. It will not be possible to prevent all faults (whether arising from security oopholes or from failure of software or hardware), and it will be difficult to mobilize a human response to faults cheaply and easily. The presence of agents can cause or amplify certain faults, and agents can be compromised by malicious users [4]. In summary, future systems are likely to use agents, and will therefore need ways to address problems caused by these agents without relying on swift human intervention. 2 Resilience – a General Approach Traditionally, overall approaches to security and reliability have either sought to prevent security lapses and faults happening in the first place, and/or to mend them through a human response when they do happen. It is generally impossible to prevent all faults, and prevention is expensive. Mending faults when they occur, on the other hand, tends to be slow, since human response time is slow compared with machine speed. Some types of fault can do considerable damage if not addressed quickly. At the height of the Code Red virus attack, Cisco’s intrusion detection system was reporting 2.5 million events a day: a fast human response to each of these events would have been infeasible. Our philosophy for reliability of complex systems, as outlined in [3], is to directly tackle the problem of damage cause by faults before a human response. The approach is to build resilient infrastructure that can hamper, mitigate and contain problems, so buying time for a human response. Since containing damage is simpler than finding the fault and deciding how to fix it, it makes sense to use automatic computer responses to contain problems, and humans to sort them out. Creating resilient systems is original as a guiding philosophy, although there are many individual instances of resilient behaviour. This approach can be applied to a very broad range of applications (not just to agents). Matthew Williamson’s virus throttling, which applies this approach, drastically slows the spread of computer viruses without noticeably affecting non-infected machines [3]. In some cases an automatic reaction might be enough to heal the fault entirely: see [7] for a discussion. In this paper we discuss applying this approach to two classes of problem behaviour by agents, looping and high-bandwidth output. 2.1 Resilient Approach to the Looping Problem One problem described in [4] is that an agent may get into an unwanted loop with its environment or with another agent. Such loops can occur for example in email mailing lists if two list members have wrongly-configured auto-reply agents that answer ach other. Following our philosophy, a generic way of addressing the looping problem for agents is to programme the agent to delay before responding again to a source (and to perform other tasks in the meantime), if the source has recently triggered a task. This slows down the loop, but does not completely close it down, so that benign loops still run – albeit slowly – and meanwhile disruption to other sources is minimized. Of course an alarm could be triggered if there is a prolonged series of rapid requests from a single source. This generic approach to loops can be refined for specific examples by categorizing requests into different types, and only delaying responses to a request from a source if there had recently been a request from the same source in the same category. For example, in the case of email loops, it is enough to delay email messages if another message was recently received from the same source to the same recipient with close to the same length (“close to” rather than “equal to” because bounce messages may contain time stamps, for example). Another refinement is for the meaning of “recently” to vary per source and/or per request type, with the associated time values derived from data on normal system behaviour.